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IMAGE AMD AUDIO DEGRADATION SIMULATOR 

Cross Reference to Related Application 

This claims the benefit of United States 
Provisional Application No. 60/197,101, filed April 12, 
2000 . 

Background of the Invention 

This invention relates to degradation of 
electrical signals representing images and audio of film and 
television productions. More particularly, this invention 
relates to real-time simulation of such signal degradation. 

The visual and audio quality of an original 
production, which may be a live or animated film or 
television production, or a live transmission, is rarely 
what an end-user (e.g., a television viewer or moviegoer) 
actually sees or hears. The visual and sound quality of an 
original production initially may be degraded because of a 
technical limitation or problem related to the electronic 
capture of the production. For example, common picture 
defects can be caused by herringbone clothing, white shirts, 
Venetian window blinds, film grain, and excessively rapid 
camera motion. Picture and sound quality of the captured 
images and audio may then be further degraded as a result of 
post -product ion processing, video processing, signal 
transmission, and display at an end-user's location. 
Post-production processing can involve editing, color 
correction, film or tape transfers, replays, storage, and 
duplication. Signal transmission can involve compression 



and decompression, analog and digital conversions, 
scrambling and descrambling, and noise and artifacts (i.e., 
stray entities that do not belong in an image) . Display at 
an end-user's location can involve low quality picture 
resolution and high levels of audio distortion. 

Many of these degradations can be cost-ef f ectively 
corrected or compensated for by making changes to the 
production while that production (e.g., a particular scene) 
is in progress. For example, simple changes to a set 
(including to background, props, colors, lighting, 
microphone positions, wardrobe, etc.) can often eliminate 
the causes or at least lessen the effects of various picture 
and sound degradations. However, a producer (or director, 
cinematographer, recording engineer, etc.), who creates the 
film or video representation of the pictures and sounds of a 
film or television production, has no way of knowing, and in 
particular, cannot see or hear the effects of various 
degradations while a production is in progress. Thus, a 
producer has no opportunity to make typically cost-effective 
on-the-spot changes that could compensate for or correct 
various degradations that will affect the quality of the 
final product seen and heard by end-users. 

Even "dailies," for example, which are film 
workprints created overnight for review in the morning to 
ensure that the previous day's scenes were filmed as 
desired, do not give a producer the opportunity to make 
immediate changes while those scenes are being filmed. At 
best, needed corrections usually involve re-assembling the 
cast and crew and re- shooting the scene on the next 
available day. Dailies are intended to be reviewed 
primarily for scene content and gross technical problems 
(e.g., inaudible dialog) while a cast, crew, and set are 
still available. Because dailies are unedited and 
unprocessed (e.g., no color correction), they are not of the 
same quality as that intended to be displayed to an 
end-user. Moreover, dailies are not intended to show the 



effects of subsequent degradations that are likely to occur. 
Thus, dailies are of limited value for making corrections 
and enhancements that would improve or maintain a desired 
level of picture and sound quality to be seen and heard by 
end-users . 

Similarly, although other picture and sound 
quality reviews are typically conducted before the film or 
videotape leaves a production facility, those reviews 
usually occur after completion of most filming or taping. 
Moreover, they also do not take into account any subsequent 
degradations that are likely to occur after the finished 
product leaves the production facility for distribution to 
theater or television operators. Again, little or no 
opportunity exists to correct or compensate for degradations 
while film or television productions are in progress. 

The same is true of known automated equipment and 
processes that operate reactively to correct various types 
of detected degradations (e.g., transmission errors, 
distortions, and other quality related problems) . These 
corrections are responsive to degradations that have already 
occurred and are not normally responsive to subsequent 
degradations that are likely to occur. Furthermore, 
correcting or compensating for some types of degradations 
may be less complex and costly had they been anticipated and 
acted upon while a production was in progress. 

In view of the foregoing, it would be desirable to 
be able to provide a degradation simulation of a 
just-completed portion of a film or television production 
that can be viewed substantially simultaneously while that 
production is in progress. 

It would also be desirable to be able to provide a 
variety of selectable degradation simulations of a 
just -completed portion of a film or television production 
that can be viewed substantially simultaneously while that 
production is in progress. 
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It would further be desirable to be able to 
provide a quantitative analysis of a degradation simulation 
of a just -completed portion of a film or television 
production that can be reviewed substantially simultaneously 
5 while that production is in progress. 

It would still further be desirable to be able to 
provide a degradation simulation that highlights 
unacceptable degradations of a just -completed portion of a 
film or television production that can be viewed 
10 substantially simultaneously while that production is in 
progress . 

Summary of the Invention 

It is an object of this invention to provide a 

degradation simulation of a just-completed portion of a film 
15 or television production that can be viewed substantially 

simultaneously while that production is in progress. 

It is also an object of this invention to provide 

a variety of selectable degradation simulations of a 

just-completed portion of a film or television production 
20 that can be viewed substantially simultaneously while that 

production is in progress. 

It is a further object of this invention to 

provide a quantitative analysis of a degradation simulation 

of a just -completed portion of a film or television 
25 production that can be reviewed substantially simultaneously 

while that production is in progress. 

It is a still further object of this invention to 

provide a degradation simulation that highlights 

unacceptable degradations of a just -completed portion of a 
30 film or television production that can be viewed 

substantially simultaneously while that production is in 

progress . 

In accordance with this invention, an image and 
audio degradation simulator is provided that includes a 
35 digital signal processor, a memory, at least one algorithm 



that performs a predetermined degradation on a received 
image or audio signal, and a display that displays images, 
audio, or both. The simulator simulates on the display in 
real time the image, audio, or both of a received signal 
5 that has been degraded in accordance with the degradation 
algorithm. Thus, for example, a producer can view a 
degradation simulation preferably substantially immediately 
while producing a scene to determine whether any re-shooting 
or production changes are needed to maintain a desired level 

10 of picture or sound quality. 

The degradation simulator of the present invention 
preferably includes a plurality of selectable degradation 
algorithms, each performing a different predetermined 
degradation. Each algorithm can be simulated individually 

is or in combination with others. The simulator also 

preferably performs recursive degradations of a selected 
algorithm. This allows a producer to simulate, for example, 
the effects of multiple signal compressions and 
decompressions on a video signal. 

20 Preferably, the degradation simulator of the 

present invention also performs a quantitative analysis of 
simulated degradations to provide one or more degradation 
measurements that can be reviewed substantially 
simultaneously with production. The degradation simulator 

25 also preferably highlights on the display in real time 

portions of a simulation in which one or more degradations 
exceed a preset threshold. 

Advantageously, the present invention can be used 
at any stage of production to simulate and evaluate the 

30 effects of degradations subsequent to that stage. 

Brief Description of the Drawings 

The above and other objects and advantages of the 
invention will be apparent upon consideration of the 
following detailed description, taken in conjunction with 



the accompanying drawings, in which like reference 
characters refer to like parts throughout, and in which: 

FIG. 1 is a simplified illustration of a 
representative film production process from creation to 
moviegoer ; 

FIG. 2 is a simplified illustration of a 
representative television production process from creation 
to television viewer; 

FIG. 3 is a block diagram of a preferred 
embodiment of a degradation simulator according to the 
invention; and 

FIG. 4 is a simplified illustration of a display 
screen with highlighted portions indicating unacceptable 
degradation according to the invention. 

Detailed Description of the Invention 

The present invention provides an image and audio 
degradation simulator that simulates in real time how images 
and audio would respectively look and sound after the 
electrical signals representing those images and audio have 
been degraded in one or more objective or subjective ways. 
In accordance with the present invention, degradation 
simulations are provided preferably substantially 
immediately upon receiving signals representing images and 
audio. Producers of film or television programming can 
advantageously simulate preferably one or more types of 
degradation on, for example, a just -completed filmed or 
taped scene or portion thereof to determine in real time 
whether any retakes or production changes are necessary -- 
while that scene is still in progress. Similarly, producers 
of live transmissions can simulate how those transmissions 
are being received by end-users and accordingly make any 
necessary production changes. Thus, a desired level of 
picture or sound quality can be maintained. 

Moreover, the degradation simulator of the present 
invention can be advantageously used at any stage of a 



production process (e.g., from creation to display at an 
end-user's location) to simulate the effects of any known 
subsequent degradations or any "what-if" degradations. For 
example, the present invention can be used with existing 
tapes and films to simulate the effects of a particular 
transmission path to an end-user. Thus, the quality of a 
particular link (e.g., a rental fiber circuit) that may be 
used for a live event can be simulated and evaluated. 
Similarly, producers can use the invention to simulate and 
evaluate a potential distributor's transmission plant before 
agreeing to use that distributor. 

The present invention does not require that both 
images and audio be present concurrently, each can be 
degraded and simulated separately, if desired. Also, the 
present invention is not limited to images of a moving 
picture, but can also be used with still images. 
Furthermore, the present invention performs no correction of 
actual or anticipated degradations, but identifies their 
effects via displayed simulations and optional quantitative 
analyses . 

Where the context permits, "displaying" includes 
displaying images, playing audio, or both, and "picture" 
includes images displayed on a movie screen, a television 
monitor, computer screen, or the like. 

FIG. l illustrates a representative film 
production process from creation to moviegoer. "Film" 
refers to motion pictures that are initially shown in 
theaters. Film production process 100 begins with the 
filming of a scene 102 with one or more film cameras 104 and 
microphones 106. Output from camera 104 is developed into a 
negative 108. Negative 108 is then used to print an 
"interpositive" 110. An interpositive is a high quality 
copy of an original negative, but with positive images, and 
is used mostly in film labs. Interpositive 110 is then 
edited and processed (e.g., different scenes and different 
camera angles of scenes are added, sequenced, and cut; color 
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correction is performed; special effects, titles, and 
credits are added; etc.). A "video assist" 112 (i.e., a 
film camera optical takeoff) is often used by producers and 
others to assist in make editing decisions. Video 
5 assist 112 is a device that provides images seen through the 
lens of film camera 104, similar to that seen through the 
eyepiece of a camera. The edited interpositive is then 
printed to form an "internegative" 114. An internegative is 
color negative of a color transparency made for purposes of 

10 duplication. 

Audio from the filmed scene is typically 
recorded on audio tape 116, and then edited and mixed to 
produce a final sound track tape 118. Editing and 
mixing 117 can include adding audio received from different 

15 microphones, dubbing inaudible portions, and adding 

background sounds. The final sound track is then usually 
etched onto internegative 114 and copied onto, for example, 
multiple compact discs (CDs) 122. The sound track may be 
formatted in SDDS (Sony Dynamic Digital Sound) . 

20 Multiple release prints 126 are then printed from 

internegative 114 and are typically distributed physically 
along with CDs 122 to theater operators. At a theater, 
release print 12 6 and CD 122 are respectively run through a 
film projector 12 8 and sound system 13 0 where pictures on 

25 screen 13 2 and sound are ultimately seen and heard by 
moviegoers . 

Degradation of images and audio can occur 
throughout process 100. For example, degradation can occur 
at video assist 112, printing 109, editing 111, 

30 printing 124, audio editing and mixing 117, audio CD 

processing 120, film projector 128, sound system 130, and 
screen 132. 

FIG. 2 illustrates a representative television 
production process from creation to television viewer. 
35 Television productions may be videotaped or live 

transmissions. Television production process 200 begins 



with the electronic capture of an event 2 02 with one or more 
television cameras 204 and microphones 206. A production 
switcher 208 controlled by an operator selects which camera 
output is transmitted or recorded. Similarly, an audio 
mixer 212 is used to select and mix the audio to be 
transmitted or recorded. For videotaping, separate tapes 
are usually made of the video and audio, both of which are 
then typically edited, assembled, and processed at 216 into 
a single production tape of the complete program. This tape 
is then played back in a videotape player 218 which feeds a 
typically automated program origination switcher 22 0. 
Switcher 22 0 integrates and assembles programming received 
from multiple sources to create the complete programming for 
a network or channel. Switcher 220 outputs the network or 
channel ' s complete programming to a program distribution 
facility 222. From there, either tapes of the programming 
are physically transported to various cable or satellite TV 
system headends 224, or the programming is transmitted to TV 
system headends 224 via satellite, the Internet, cable 
(e.g., coax or fiber optic), or any other suitable medium. 
During such transmission, programming is typically 
digitized, compressed, and possibly scrambled. From 
headend 224, the program is transmitted through the system 
plant 226 to one or more set-top boxes or televisions 228 
where the final product can be seen and heard by television 
viewers . 

The live transmission path is very similar, except 
that instead of creating a tape that is edited and played 
back in a videotape player, the electrical signals output 
from production switcher 2 08 and audio mixer 212 are 
transmitted directly to program origination switcher 22 0 via 
satellite, cable (e.g., coax or fiber optic), the Internet, 
or any other suitable medium. 

Similar to process 100, degradation of electrical 
signals representing images and audio of a television 
production can occur at any stage. For example, degradation 
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can occur at cameras 204, switcher 208, editing 216, 
playback 218, program origination switcher 220, program 
distribution 222, TV system headend 224, system plant 22 6, 
and set-top box/television 228. 
5 Hybrid film/television processes that include 

elements of both processes 100 and 200 are also common. For 
example, film productions can sometimes be produced less 
expensively by following process 2 00 up to playback 218 
where, instead of feeding the output of playback 218 into 

10 program origination switcher 22 0, the output of playback 218 
is used to create an internegative 114 . Similarly, when 
films are to be shown on television, the film's 
internegative 114 is processed with a "telecine" device that 
transfers film to videotape. That videotape is then played 

15 at playback 218 and fed into program origination 

switcher 22 0. Each of these hybrid processes incurs the 
same types of signal degradations as processes 100 and 200. 

Although some degradation in picture and sound 
quality between an original production and what end-users 

20 see and hear is expected and usually imperceptible, 

objectionable undetected degradations can and do occur, 
reaching as far as the end-user. These degradations often 
can be corrected or compensated for had they been known or 
anticipated during production. 

25 PIG. 3 shows a preferred embodiment of an image 

and audio degradation simulator 300 in accordance with the 
present invention. Simulator 300 includes input 302 
and output 304, and preferably includes input 306 and 
output 308. Input 302 receives digital signals, while 

30 input 3 06 receives analog signals. These signals represent 
images and audio of preferably film, television, or hybrid 
productions. Output 3 04 provides a degraded digital signal, 
while output 308 provides a degraded analog signal. 
Preferably, simulator 300 includes a conventional 

35 analog-to-digital (A/D) converter 310 for converting analog 
input signals to digital signals, and a conventional 
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digital-to-analog (D/A) converter 312 for converting digital 
output signals to analog signals. The analog output signals 
can be used for subsequent processing, recording, or quality 
control logging. 

Input signal sources to simulator 3 00 can include, 
for example, video assist devices, film transfers (e.g., 
"dailies," interpositives , internegatives , and release 
prints) , video camera sources and production switchers, 
(e.g., high-quality video and RGB (red-green-blue)), 
microphones, audio mixing console, and analog sources (e.g., 
cable headend transmissions or receptions and satellite or 
cable (e.g., coax or fiber optic) transmissions or 
receptions) . Digital signals received at input 302 are 
preferably in a serial digital component format, such as, 
for example, Serial Data Interconnect (SDI) . SDI includes 
both digital video and embedded audio. Alternatively, where 
only analog picture, sound, or both are available, those 
signals can be coupled to input 3 06 for conversion into 
digital form by analog- to-digital converter 310. 

Simulator 3 00 also includes a digital signal 
processor (DSP) 314, which can be a personal computer. 
DSP 314 includes a central processing unit (CPU) and 
preferably includes a RAM- storage video device, such as, for 
example, a ViewStore 6000 by Viewgraphics Incorporated, 
which has been acquired by Optibase, Inc., of San Jose, 
California. This device provides both a frame-buffering 
function and access to the CPU. The CPU performs 
algorithmic functions on the received image and audio 
signals. DSP 314 preferably includes a Pentium® III 
processor by Intel Corporation, of Santa Clara, California, 
or equivalent . 

Simulator 300 further includes a memory 316 of 
preferably 128 megabytes. Although shown separately 
(primarily for illustrative purposes) , memory 316 can be 
incorporated within DSP 314 and is preferably random access 
memory. Memory 316 can additionally include, or can 
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alternatively be, other types of suitable memory such as 
EE PROM, EPROM, disk, or tape. Digital signals received by 
DSP 314 are stored in memory 316. 

Simulator 3 00 includes at least one and preferably 
5 a plurality of signal degradation algorithms 318 that can be 
stored in memory 316 or other suitable memory or storage 
coupled to DSP 314. Each algorithm 318 performs a 
predetermined signal degradation on a received digital 
signal. Algorithms 318 preferably perform both objective 

10 and subjective signal degradations. Preferably, they 

include the following types of film degradations: picture 
resolution loss (of preferably various degrees) , "gate 
weave," and "shutter judder." Algorithms 318 preferably 
perform the following types of television film degradations: 

15 three-two pulldown motion artifacts; NTSC (National 

Television Standards Committee) "cross -chroma, reduced color 
gamut, cross-luma, and interlace 'twitter,-'" PAL (phase 
alternate line) "speed variation (audio) and interlace 
•twitter.'" NTSC is a North American broadcast standard 

20 specifying, among other things, 525 lines in each complete 
video frame. PAL is a European color TV standard 
specifying, among other things, 62 5 lines in each complete 
video frame. Algorithms 318 also preferably perform the 
following types of television video degradations: NTSC/PAL 

25 artifacts as mentioned above, satellite FM/ triangular noise 
spectrum and IF (intermediate frequency) truncation, 
analog-to-digital conversion, digital-to-analog conversion, 
VSB-AM (vestigial sideband - amplitude modulation) 
television transmission artifacts, MPEG (Moving Pictures 

30 Expert Group) signal compressions (of various bit rates and 
other parameters such as 4:2:2/4:2:0), cable television 
"cascade, 'triple-beat, ' and 'composite second-order 
distortion.'" Algorithms 318 further preferably perform the 
following types of audio degradations: analog-to-digital 

35 conversion, noise reduction, sub-band coding and compression 
artifacts, and digital-to-analog conversion. 



Algorithms 318 preferably include DSP digital 
filtering taps and memory operations that operate on the 
received image and audio signals. Simulator 300 can execute 
algorithms 318 individually, recursively, or in combination 
5 with each other, as selected by a user. Memory 316 

preferably includes lookup tables for algorithms 318 to 
facilitate their loading and execution. 

Alternatively, in another embodiment of 
simulator 300 in accordance with the present invention, the 
10 degradation functions performed by algorithms 318 can be 
performed instead by dedicated hardware in DSP 314. 

Simulator 300 preferably includes an optional 
analytical processor 320 coupled to outputs 321 and 322 of 
DSP 314. Output 321 provides a digital signal received by 
15 DSP 314 before any degradations are performed on it, while 
output 322 provides that digital signal degraded in 
accordance with one or more selected algorithms 318. 
Analytical processor 32 0 analyzes received signals to 
provide objective measurements of the encountered signal 
20 degradation. Analytical processor 32 0 preferably includes a 
picture quality analyzer, such as, for example, a Model 
PQA300 by Tektronix, Inc., of Beaverton, Oregon. Picture 
degradation measurements preferably include signal-to-noise 
ratios and frequency responses. Analytical processor 320 
25 also preferably includes an audio analyzer, such as, for 

example, a System One or System Two series audio analyzer by 
Audio Precision, also of Beaverton, Oregon. An audio 
analyzer provides an equivalent set of objective degradation 
measurements. These degradation measurements are 
30 displayable on a display. Alternatively or additionally, 
analytical processor 32 0 may include its own display for 
displaying degradation measurements. 

Simulator 3 00 includes display 324, which can be, 
for example, a 14" Model BVM-D14H1A or 20" Model BVM2 0E1A 
35 broadcast monitor by Sony Corporation of America, of New 
York, NY. Display 324 preferably displays the following 
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types of outputs: picture and sound as degraded by one or 
more selectable algorithms 318, showing what the 
degradations incurred along a particular film, television, 
or hybrid production path would do to a picture or sound 
5 track; analytical measurements of degradations incurred 
along a selected production path including, for example, 
"depth of degradation" (e.g., percent of defect), signal-to- 
noise ratio, root sum square (RSS) of accumulated 
degradations (e.g., ANSI/EIA RS-250 accumulations); and an 
10 enhanced display including, for example, a display with 
"zebra stripes" highlighting a particular defect (e.g., a 
Venetian blinds picture defect) . 

Analytical processor 32 0 preferably executes 
software 326 that preferably uses degradations measurements 
15 and preset thresholds to provide highlighting (e.g., 

brightened, crosshatched, or colored areas) superimposed 
over portions of a display where degradation (e.g., a 
signal-to-noise ratio related to a picture defect) exceeds a 
preset threshold. For example, software 32 6 may provide 
20 crosshatching superimposed over a displayed portion of rapid 
camera motion when the picture complexity exceeds the 
ability of the data-rate-constrained MPEG compression 
decoder to faithfully reproduce the picture above a preset 
subjective threshold set by a system operator (e.g., the 
25 producer) . To illustrate, FIG. 4 shows display 324 in which 
portions of a degradation simulation of a recorded or filmed 
scene are highlighted with crosshatching 44 0, indicating 
that the degradations in those areas exceed a preset 
threshold. 

30 Described below are particular details of a 

simulation by simulator 300 of degradation resulting from a 
single pass of MPEG digital video compression and MPEG 
Layer 3 audio compression (e.g., MUSICAM) . The simulation 
assumes a signal representing images and audio being 

35 transmitted via a digital cable-TV path to a typical set-top 
box that includes a consumer NTSC interface using VSB analog 



modulation at channel 3 to a consumer- type display (e.g., a 
conventional television screen) . 

For MPEG video degradation, an algorithm 318 
preferably includes a discrete cosine transform (DCT) to 
show the impacts of the imperfect frequency reproductions in 
the early stages of MPEG compression processing. Real-time 
inputs to algorithm 318 include the likely target MPEG bit 
rate and picture resolution of the transmitted picture. 
Other algorithms 318 (or one that performs all mentioned 
degradations) are also executed to simulate the MPEG decoder 
(e.g., filter characteristics such as group -del ay ) , NTSC 
conversion (e.g., band-limited chroma, cross-color, etc.), 
VSB modulation (e.g., pre-shoots, filter ringing, etc.), and 
the consumer display (e.g., resolution impacts of screen 
dot-pitch, display phosphors, etc.). 

Similarly, received audio signals are processed by 
one or more algorithms 318 that include filter tap points 
and filter characteristics for simulating what would be 
incurred in the audio sub-band companded processor (e.g., 
MUSICAM) . Companding refers to signal compression and 
expansion. Real-time data inputs provide algorithms 318 
with the data rate and version of the MUSICAM encoder and 
decoder to be simulated. Additional algorithms 318 can also 
be selected, preferably with a graphical user interface 
(GUI) preferably included in simulator 3 00, to include 
loudspeaker fidelity, amplifier distortion, or other gross 
or subtle degradations. The GUI permits a 'chain' of 
algorithms 318 to be executed for simulating different types 
of degradations that may be incurred along a process path 
(including recursive executions of particular algorithms 318 
that can simulate, for example, the effects of multiple 
passes of MPEG encoding, A/D and D/A conversions, etc.) . 

Algorithms 318 preferably perform the following 
degradations on received signals: 



Video degradations: 

MPEG-2 encoding (version: Motorola DigiCipher , TSP 
version 7) ; 

Target bit rate: 2-6 Mb/sec, variable bit rate, 
statistical multiplexing, medium load; 
resolution: 3/4; 

target transmission path: cable TV digital QAM 
(quadrature amplitude modulation) , BER (bit error 
rate) > 1(T 4 ; 

target decoder: cable set top box, Motorola 
DCT-1000; and 

display: consumer TV grade dot pitch, NTSC 
compatible via channel 3 modulator. 

Audio degradations : 

MPEG Layer 3 audio compression (version: MUSICAM; 
Philips version 2.0); 

Target bit rate: 12 8 kb/sec, network congestion: 
light; and 

Speaker: consumer grade 5 -inch or smaller 
loudspeaker; 2% THD (distortion) . 

In a continuous stream, the digital video and 
audio degraded in accordance with selected algorithms 318 
are provided at output 3 04 for display on display 324 and 
preferably at output 322 for quantitative analysis at 
analytical processor 32 0, as described above. 

Thus it is seen that an image and audio 
degradation simulator is presented that simulates in real 
time selected degradations of images and sound of film, 
television, or hybrid productions either while those 
productions are being electronically captured or at anytime 
thereafter. One skilled in the art will appreciate that the 
present invention can be practiced by other than the 
described embodiments, which are presented for purposes of 
illustration and not of limitation, and the present 
invention is limited only by the claims which follow. 



